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IN THE CLAIMS 

1. (Currently Amended) A computer readable medium having 
instructions, which when implemented by a computer generate client 
side markup for a client in a server/client system, the 
instructions comprising: 

a set of modality dependent controls d e fined in an authoring 
page for a website for defining desired visual 
renderings and at least one of recognition and audible 
prompting on a client in a server/client system, each 
control having a first set of attributes directly 
related to visual rendering and a second set of 
attributes directly related to at least one of 
recognition and audibly prompting, the controls being 
related to client side markup executable by a client 
browser; and 

a module operable on a computer, the module being configured 

to receive anRb -the authoring page - -- ger - a wob g i t c 

compriei r H - g ' a--p - lurQlity of controls , and wherein the 

module is further configured to prooees the r controlo to 

generate , using modality dependent attributes provided 
directly from the controls on the authoring page, 
client side markup executable by the client browser on 
the client in the server/client system in accordance 
with the controls and the attributes of the controls to 
perform both visual rendering and at least one of 
recognition and audibly prompting. 

2. {Previously Presented) The computer readable medium of claim 
1 wherein one of the second set of attributes for one of the 
controls relates to a grammar to use for recognition. 

3* {Previously Presented) The computer readable medium of claim 
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2 wherein said one of the second set of attributes provides a 
reference to a location of the granomar. 

4. {Previously Presented) The computer readable medium of claim 
2 wherein the grammar is for one of speech recognition, 
handwriting recognition, gesture recognition and visual 
recognition. 

5. (Previously Presented) The computer readable medium of claim 
4 wherein the controls relate to one of HTML, XHTML, cHTML, XML 
and WML. 

6. (Previously Presented) The computer readable medium of claim 
1 wherein the controls relate to one of HTML, XHTML, cHTML, XML 
and WML. 

1, (Previously Presented) The computer readable medium of claim 
1 wherein one of the second set of attributes for one of the 
controls provides instructions related to generating audible 
output • 

8. (Previously Presented) The computer readable medium of claim 
7 wherein the instructions comprise text and the attribute relates 
to converting the text to audible output, 

9* {Previously Presented} The computer readable medium of claim 
1 wherein one of the second set of attributes for one of the 
controls relates to a location of data for audible output. 

10. (Previously Presented) The computer readable medium of claim 
9 wherein the data comprises a prerecorded audio data file and the 
attribute relates to playing the prerecorded audio data file* 
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11. (Previously Presented) The computer readable medium of claim 
9 wherein the data comprises text and the attribute relates to 
converting the text to audible output, 

12* (Currently Amended) A computer readable medium having 
instructions defined on an authoring page for a website , which 
when implemented by a computer generate client side markup for a 
client in a server /client system, the instructions comprising: 
a first set of modality dependent visual controls 

having attributes directly related to defining 
desired visual renderings on the client device^ 
the first set of controls being related to client 
side markup executable by a client browser ^ and 
wherein the attributes include at least one of 
location for visual rendering^ font, background 
color, and foreground color ; and 
a second set of modality dependent controls having 

attributes directly related to defining desired 
operation on the client device comprising at least 
one of recognition and audibly prompting, the 
second set of controls using at least one of the 
first controls, the second set of controls being 
related to client side markup executable by a ■ 
client browser; and 
a module operable on a computer, the module being 

configured to receive a ^the authoring page, which 

includes for a webgito compriGing a plurality of 

the second set of controls, wherein the module is 
further configured to process the plurality of the 
second set of controls from the authoring page to 
generate client side markup directly from the 
modality dependent controls that is executable by 
the client browser on the client in the 
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server/client system in accordance with the second 
set of controls and the attributes of the second 
set of controls for at least one of recognition 
and audibly prompting, and wherein the module is 
configured to use at least one of the first set of 
controls from the authoring page in order to 
generate markup directly therefrom when processing 
each of the second set of controls. 

13. (Previously Presented) The computer readable medium of claim 

12 wherein one of the attributes for the second set of controls 
relates to a grammar to use for recognition, 

14. (Previously Presented) The computer readable medium of claim 

13 wherein said one of the attributes provides a reference to a 
location of the grammar. 

15. (Previously Presented) The computer readable medium of claim 
13 wherein the grammar is for one of speech recognition, 
handwriting recognition, gesture recognition and visual 
recognition. 

16. {Previously Presented) The computer readable medium of claim 
15 wherein the first set of controls and the second set of 
controls relate to one of HTML, XHTML, cHTML, XML and WML. 

17. (Previously Presented) The computer readable medium of claim 
12 wherein the controls relate to one of HTML, XHTML, cHTML, XML 
and WML. 

18. (Previously Presented) The computer readable medium of claim 
12 wherein one of the second set of attributes for one of the 
controls provides instructions related to generating audible 
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output . 

19. (Previously Presented) The computer readable medium of claim 
18 wherein the instructions comprise text and the attribute 
relates to converting the text to audible output* 

20. {Previously Presented) The computer readable medium of 
claim 12 wherein one of the second set of attributes for one of 
the controls relates to a location of data for audible output. 

21. (Previously Presented) The computer readable medium of claim 
20 wherein the data comprises a prerecorded audio data file and 
the attribute relates to playing the prerecorded audio data file. 

22. (Previously Presented) The computer readable medium of claim 
20 wherein the data comprises text and the attribute relates to 
converting the text to audible output. 

23. (Currently Amended) A computer readable medium having 
instructions, which when implemented by a computer generate client 
side markup for a client in a server/client system, the 
instructions comprising; 

a first set of modality dependent visual controls 

defined on an authoring page for a website having 
attributes directly related to defining desired 
visual renderings on the client device, the first 
set of controls being related to client side 
markup executable by a client browser; and 

a second set of modality dependent controls defined on 
the authoring p age for having attributc o dir eefely 
related to defining desired operation on the 

client device comprioing having attributes 

directly related to at least one of recognition 
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and audibly prompting audible prompting , wherein 
attributes directly related to recognition include 
at least one of location of grammar for use in 
recognition and confidence level thresholds for 
recognition and wherein attributes directly 
related to audible prompting include at least one 
of inline text for text-to-speech conversion, 
location of data for audible rendering and playing 
of a prerecorded audio file, the second set of 
controls being selectively associated with the 
first controls, and the second set of controls 
being related to client side markup executable a 
client browser; and 
a module operable on a computer, the module being 

configured to receive afr -the authoring page for a 

wobsitc comprioing a - plurality of oont - rola of the 

first act and a plurality of controlo of the 
eeeond oct , wherein the module is further 
configured to process the controls of the first 
set in the authoring page to generate client side 
markup by incorporating the attributes in the 
modality dependent controls that is executable by 
the client browser on the client in the 
server/client system in accordance with the 
controls of the first set and the attributes of 
the controls of the first set to perform both 
visual rendering, and wherein the module is 
further configured to process the controls of the 
second set to generate client side markup by 
incorporating the attributes in the modality 
dependent controls that is executable by the 
client browser on the client in the server/client 
system in accordance with the controls of the 



second set and the attributes of the controls of 
the second set in the authoring page to perfom at 
least one of recognition and audibly prompting. 

24. (Previously Presented} The computer readable medium of claim 

23 wherein one of the attributes for the second set of controls 
relates to a grammar to use for recognition, 

25. (Previously Presented) The computer readable medium of claim 

24 wherein said one of the attributes provides a reference to a 
location of the grammar. 

26. {Previously Presented) The computer readable medium of claim 
24 wherein the grammar is for one of speech recognition, 
handwriting recognition, gesture recognition and visual 
recognition. 

27. (Previously Presented) The computer readable medium of claim 
26 wherein the first set of controls and the second set of 
controls relate to one of HTML, XHTML, cHTML, XML and WML. 

28. {Previously Presented) The computer readable medium of claim 
23 wherein the controls relate to one of HTML, XHTML, cHTML, XML 
and WML. 

29. {Previously Presented) The computer readable medium of 
claim 23 wherein one of the second set of attributes for one of 
the controls provides instructions related to generating audible 
output . 

30. (Previously Presented) The computer readable medium of claim 
29 wherein the instructions comprise text and the attribute 
relates to converting the text to audible output. 
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31, (Previously Presented) The computer readable medium of claim 
23 wherein one of the second set of attributes for one of the 
controls relates to a location of data for audible output, 

32. (Previously Presented) The computer readable medium of claim 
31 wherein the data comprises a prerecorded audio data file and 
the attribute relates to playing the prerecorded audio data file- 

33- (Previously Presented) The computer readable medium of claim 
31 wherein the data comprises text and the attribute relates to 
converting the text to audible output. 

34. (Previously Presented) The computer readable medium of claim 
23 wherein one of the attributes of the second set of controls 
relates to an identifier of the associated control of the first 
set of controls to form the association, 

35. (Previously Presented) The computer readable mediuna of claim 
23 wherein one of the attributes relates to whether the associated 
control of the second set is available for activation. 

36. (Previously Presented) The computer readable medium of claim 
35 wherein activation relates to generating markup. 

37. (Previously Presented) The computer readable medium of claim 
35 wherein activation relates to execution on the client. 

38. (Previously Presented) The computer readable medium of claim 
23 wherein the second set of controls activates another control of 
the second set , 

39. (Previously Presented) The computer readable medium of claim 
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23 wherein the second set of controls comprise: 

a question control for generating markup related to audible 

prompting of a question; and 
an answer control for generating markup related to a grammar 
for recognition , 

40* (Previously Presented) The computer readable medium of claim 

39 wherein the question control activates the answer control- 

41- (Previously Presented) The computer readable medium of claim 

40 wherein the answer control includes a mechanism to associate a 
received result with one of the first set of controls* 



42. (Previously Presented) The computer readable medium of claim 

41 wherein mechanism includes binding the recognition value* 

43- (Previously Presented) The computer readable medium of claim 

42 wherein the mechanism includes issuing an event related to 
operation of binding, 

44* (Previously Presented) The computer readable medium of claim 

40 wherein the second set of controls comprise: 

a command control for generating markup related to a 

grammar for one of navigation in the markup, help 
with a task, and repeating an audible prompt. 

45, (Previously Presented) The computer readable medium of claim 
40 wherein the second set of controls comprise: 

a confirmation control for generating markup related to 
confirming that a recognized result is correct* 



46. (Previously Presented) The computer readable medium of claim 
45 wherein the confirmation control includes an attribute related 
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to the recognized result to be confirmed- 

47. (Previously Presented) The computer readable medium of claim 
46 wherein the answer control includes an attribute related to a 
confidence level. 

48. {Previously Presented) The computer readable medium of claim 
46 wherein the confirmation control is activated as a function of 
a confidence level of a received result. 

49. (Previously Presented) The computer readable medium of claim 
48 wherein the confirmation control activates an accept control to 
accept the recognized result. 

50. (Previously Presented) The computer readable medium of claim 
48 wherein the confirmation control activates a deny control to 
deny the recognized result* 

51. (Previously Presented) The computer readable medium of claim 
48 wherein the confirmation control activates a correct control to 
correct the recognized result. 

52. (Currently Amended) A computer implemented method for 
defining a website application on a server in a server /client 
architecture ;r the website application providing markup to a client 
for performing recognition and/or audible prompting on the client;, 
the method comprising: 

defining the website application by creating an 
authoring page with a first set of modality 
dependent visual controls having attributes 
directly related to visual rendering on the client 
device , the attributes directly related to visual 
rendering inclu di ng at least one of location for 
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visual rendering, font^ background color^ and 
foreground color and with a second set of modality 
dependent controls having attributes directly 
related to at least one of recognition and audibly 
prompting, the first set of controls and the 
second set of controls being related to client 
side markup executable by a client browser, 
wherein defining includes selectively associating 
controls of the second set of controls with at 
least one control of the first set of visual 
controls; and 
processing the controls on the authoring page to 

directly generate client side markup including 
processing of the first set of visual controls to 
generate client side markup executable by the 
client browser on the client in the server/client 
system in accordance with the modality dependent 
controls of the first set and the attributes of 
the controls of the first set to perform both 
visual rendering, and w - ke - r - e - i - n— t - he--mo du 1 o i g 
further configured to process ing the controls of 
the second set to generate client side markup by 
incorporating the attributes i n the modality 
dependent controls that is executable by the 
client browser on the client in the server/client 
system in accordance with the controls of the 
second set and the attributes of the controls of 
the second set to perform at least one of 
recognition and audibly prompting. 



53. (Previously Presented) The computer implemented method of 
claim 52 wherein each of the controls of the second set include an 
identifier attribute for identifying a control of the first set of 
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visual controls, and wherein associating includes providing an 
identifier of at least one control of the first set of controls in 
the corresponding identifier attribute of each of the second set 
of controls* 

54. (Previously Presented) The computer implemented method of 
claim 52 wherein the second set of controls includes a question 
control related to audible prompting of a question, and an answer 
control related to a grammar for recognition; and wherein defining 
the website application with a second set of controls related to 
at least one of recognition and audibly prompting includes 
associating the answer control with the question control - 

55. (Previously Presented) The computer implemented method of 
claim 54 wherein the second set of controls includes a 
confirmation control related to confirming that a recognized 
result is correct; and wherein defining the website application 
with a second set of controls related to at least one of 
recognition and audibly prompting includes associating the 
confirmation control with a recognized result to be received, 

56. (Previously Presented) The computer implemented method of 
claim 55 wherein the second set of controls includes a command 
control for generating markup related to a grammar for one of 
navigation on the computer, help with a task, and repeating an 
audible prompt; and wherein defining the website application with 
a second set of controls related to at least one of recognition 
and audibly prompting includes associating the command control 
with a question control. 



